Speech-enabled Card Games for Language Learners

This paper debuts a novel application of speech recognition to foreign language learning. We present a generic framework for developing user-customizable card games designed to aid learners in the difficult task of vocabulary acquisition. We also describe a prototype game built on this framework that, using a Mandarin speech recognizer, provides a student of Chinese with opportunities to speak vocabulary items in a meaningful context. The system dynamically loads only the necessary vocabulary for each game in an effort to maintain robust recognition performance without limiting the lexical domain. To assess the Sentence Error Rate (SER) of our prototype, we asked college-age students from various universities in the United States and beyond to participate in aWeb-based user study. The three central concepts of the game were recognized with a SER of 16.02%, illustrating the feasibility of deploying this system in a university curriculum via the Internet. Finally, to ensure that our recognizer is behaving appropriately with regard to learner speech, we perform a rigorous analysis of the recognition errors to determine their underlying causes.